Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ML] Natural Language Processing tasks and models #73523

Merged
merged 21 commits into from
Jun 2, 2021
Merged

[ML] Natural Language Processing tasks and models #73523

merged 21 commits into from
Jun 2, 2021

Conversation

davidkyle
Copy link
Member

@davidkyle davidkyle commented May 28, 2021 •
edited by dimitris-athanasiou
Loading

Following on from #72218 which defined how large PyTorch models can be stored, this PR introduces the concepts of Natural Language Processing tasks and defines a way to evaluate BERT models.

Mask Fill and Named Entity Recognition tasks are implemented here but others could be easily added now the framework is in place. In particular this PR implements tokenisation of input text for BERT models and defines a structure for post-graph processing.

Once the PyTorch model is uploaded a trained model config referencing it must be PUT

PUT ml/trained_models/bert-model-for-maskfill
{
    "description": "Mask fill model",
    "model_type": "pytorch",
    "inference_config": {
        "classification": {
            "num_top_classes": 1
        }
    },
    "input": {
        "field_names": ["text_field"]
    },
    "location": {
        "index": {
            "model_id": "bert-model-for-maskfill",
            "name": "big_model"
        }
    }
}

And the model deployed:

POST _ml/trained_models/deployment/bert-model-for-maskfill/_start

Mask Fill Example

POST _ml/trained_models/deployment/bert-model-for-maskfill/_infer
{
  "input": "Paris is the [MASK] of France."
}

Returns

[
  {
    "token" : "capital",
    "score" : 0.9861745037766138,
    "sequence" : "Paris is the capital of France."
  },
  {
    "token" : "center",
    "score" : 0.00372138405614492,
    "sequence" : "Paris is the center of France."
  },
  {
    "token" : "Capital",
    "score" : 0.003259749401778711,
    "sequence" : "Paris is the Capital of France."
  },
  {
    "token" : "centre",
    "score" : 0.002157122475609145,
    "sequence" : "Paris is the centre of France."
  },
  {
    "token" : "city",
    "score" : 9.026127599384262E-4,
    "sequence" : "Paris is the city of France."
  }
]

NER Example

POST _ml/trained_models/deployment/bert-model-fine-tuned-for-ner/_infer
{
  "input": "Today's GAH is live from Amsterdam, BC, London, Munich and Texas"
}

Returns:

[
  {
    "label" : "organisation",
    "score" : 0.940775243737086,
    "word" : "GAH"
  },
  {
    "label" : "location",
    "score" : 0.9987588832004948,
    "word" : "Amsterdam"
  },
  {
    "label" : "location",
    "score" : 0.9958452874139202,
    "word" : "BC"
  },
  {
    "label" : "location",
    "score" : 0.9981461858828271,
    "word" : "London"
  },
  {
    "label" : "location",
    "score" : 0.9991212183928049,
    "word" : "Munich"
  },
  {
    "label" : "location",
    "score" : 0.9994121461792658,
    "word" : "Texas"
  }
]

Feature branch PR

Co-authored-by: Dimitris Athanasiou [email protected]

@davidkyle davidkyle added >feature :ml Machine learning labels May 28, 2021
@elasticmachine elasticmachine added the Team:ML Meta label for the ML team label May 28, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@benwtrent
Copy link
Member

run elasticsearch-ci/part-1

@mark-vieira
Copy link
Contributor

jenkins test this please

@davidkyle davidkyle mentioned this pull request Jun 1, 2021
Closed
dimitris-athanasiou
dimitris-athanasiou reviewed Jun 1, 2021
View reviewed changes
Copy link
Contributor

@dimitris-athanasiou dimitris-athanasiou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just a couple of test related comments.

x-pack/plugin/ml/src/test/java/org/elasticsearch/xpack/ml/inference/nlp/TaskTypeTests.java Outdated
import org.elasticsearch.test.ESTestCase;


public class TaskTypeTests extends ESTestCase {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is left empty. Should we add some tests here?

x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/inference/nlp/FillMaskProcessor.java
import java.util.List;
import java.util.stream.Collectors;

public class FillMaskProcessor implements NlpTask.Processor {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add some tests for this one

dimitris-athanasiou
dimitris-athanasiou approved these changes Jun 1, 2021
View reviewed changes
Copy link
Contributor

@dimitris-athanasiou dimitris-athanasiou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Just a question about the name of the fill mask results field. Good to merge though even if you decide to change that.

...in/core/src/main/java/org/elasticsearch/xpack/core/ml/inference/results/FillMaskResults.java
@@ -26,25 +26,25 @@
public static final String NAME = "fill_mask_result";
public static final String DEFAULT_RESULTS_FIELD = "results";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also be predictions?

x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/inference/nlp/FillMaskProcessor.java
@@ -27,40 +26,49 @@
this.bertRequestBuilder = new BertRequestBuilder(tokenizer);
}

@Override
public void validateInputs(String inputs) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@davidkyle davidkyle merged commit 8e51034 into elastic:feature/pytorch-inference Jun 2, 2021
@davidkyle davidkyle deleted the bert-tokenizer branch June 2, 2021 10:13
@davidkyle davidkyle mentioned this pull request Jun 2, 2021
Merged
davidkyle added a commit that referenced this pull request Jun 3, 2021
@davidkyle
[ML] Merge the pytorch-inference feature branch (#73660)
94adaa5 
The feature branch contains changes to configure PyTorch models with a 
TrainedModelConfig and defines a format to store the binary models. 
The _start and _stop deployment actions control the model lifecycle 
and the model can be directly evaluated with the _infer endpoint. 
2 Types of NLP tasks are supported: Named Entity Recognition and Fill Mask.

The feature branch consists of these PRs: #73523, #72218, #71679
#71323, #71035, #71177, #70713
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dimitris-athanasiou dimitris-athanasiou dimitris-athanasiou approved these changes

Assignees
No one assigned
Labels
>feature :ml Machine learning Team:ML Meta label for the ML team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@davidkyle @elasticmachine @benwtrent @mark-vieira @dimitris-athanasiou